Skip to content

fix: Collection deletion - Adding Cumulus Execution Deletions#597

Merged
ngachung merged 14 commits intodevelopfrom
collection-delete
Jun 25, 2025
Merged

fix: Collection deletion - Adding Cumulus Execution Deletions#597
ngachung merged 14 commits intodevelopfrom
collection-delete

Conversation

@wphyojpl
Copy link
Collaborator

@wphyojpl wphyojpl commented Jun 10, 2025

No description provided.

@wphyojpl wphyojpl changed the title fix: Collection delete fix: Collection deletion - Adding Cumulus Execution Deletions Jun 10, 2025
@wphyojpl
Copy link
Collaborator Author

wphyojpl commented Jun 10, 2025

Getting this error.

    {
        "message": {
            "cumulus_executions_deletion": {
                "statusCode": 500,
                "body": {
                    "message": {
                        "client_error": {
                            "statusCode": 400,
                            "body": "{\"statusCode\":400,\"error\":\"Bad Request\",\"message\":\"collectionId URN___NASA___UNITY___UDS_LOCAL_TEST_3___DEV___DDD___01___20250610055935 is invalid\"}",
                            "headers": {
                                "x-powered-by": "Express",
                                "access-control-allow-origin": "*",
                                "strict-transport-security": "max-age=31536000; includeSubDomains",
                                "content-type": "application/json; charset=utf-8",
                                "content-length": "148",
                                "etag": "W/\"94-1A/84xKXF3FHSFHo5cVOw9BZPyE\"",
                                "date": "Tue, 10 Jun 2025 13:00:47 GMT",
                                "connection": "keep-alive",
                                "keep-alive": "timeout=5"
                            },
                            "isBase64Encoded": false
                        }
                    }
                }
            },
            "cumulus_rule_deletion": {
                "status": "Record deleted"
            },
            "cumulus_collection_deletion": {
                "status": "Record deleted"
            },
            "uds_collection_deletion": "succeeded"
        }
    },

@ngachung
Copy link
Collaborator

ngachung commented Jun 10, 2025

Why are we replacing colon in the id with ___?
https://github.com/unity-sds/unity-data-services/pull/597/files#diff-9bd5257835b713e139cc5fa1402a7a4a45f70912f77889e24a0296db304ae424R199
The collection name stored in Cumulus collection table has the colon. I believe it's just the rule names that can't have colon.

@wphyojpl
Copy link
Collaborator Author

wphyojpl commented Jun 10, 2025

Oh.. I confused with the Rule names where I'm doing that for Rule name.
Let me update and try again

@wphyojpl
Copy link
Collaborator Author

wphyojpl commented Jun 10, 2025

Still the same error.

[
    {
        "message": {
            "cumulus_executions_deletion": {
                "statusCode": 500,
                "body": {
                    "message": {
                        "client_error": {
                            "statusCode": 400,
                            "body": "{\"statusCode\":400,\"error\":\"Bad Request\",\"message\":\"invalid collectionId: \\\"URN:NASA:UNITY:UDS_LOCAL_TEST_3:DEV:DDD-01\\\"\"}",
                            "headers": {
                                "x-powered-by": "Express",
                                "access-control-allow-origin": "*",
                                "strict-transport-security": "max-age=31536000; includeSubDomains",
                                "content-type": "application/json; charset=utf-8",
                                "content-length": "121",
                                "etag": "W/\"79-1YbDnDhcVqfPTdiyQQ38pZxEW1E\"",
                                "date": "Tue, 10 Jun 2025 23:05:34 GMT",
                                "connection": "keep-alive",
                                "keep-alive": "timeout=5"
                            },
                            "isBase64Encoded": false
                        }
                    }
                }
            },
            "cumulus_rule_deletion": {
                "status": "Record deleted"
            },
            "cumulus_collection_deletion": {
                "status": "Record deleted"
            },
            "uds_collection_deletion": "succeeded"
        }
    },
    200
]

Is it possible because it doesn't have any executions?

@ngachung
Copy link
Collaborator

ngachung commented Jun 10, 2025

Cumulus doc says "collectionId": "COLLECTION_NAME___COLLECTION_VERSION"
I think you're missing version in the most recent request.

@wphyojpl
Copy link
Collaborator Author

wphyojpl commented Jun 10, 2025 

[
    {
        "message": {
            "cumulus_executions_deletion": {
                "statusCode": 500,
                "body": {
                    "message": {
                        "server_error": "invalid response: {'id': '5f567747-985b-4463-a18f-ee5374ccb062'}"
                    }
                }
            },
            "cumulus_rule_deletion": {
                "status": "Record deleted"
            },
            "cumulus_collection_deletion": {
                "status": "Record deleted"
            },
            "uds_collection_deletion": "succeeded"
        }
    },
    200
]

New Error.

@ngachung
Copy link
Collaborator

ngachung commented Jun 10, 2025 •
edited
Loading

Error in cloudwatch logs async-operation/AsyncOperation/606e2c01681b44449175f66e9db06194

Batch size is too large, size must be less than or equal to: [10000] but was [100000]

Since these are all issues we encounter when deleting the collection from Cumulus. If we only delete from DS catalog, will search results still return the collection? I'm wondering if we should just leave the collection in Cumulus.

@wphyojpl
Copy link
Collaborator Author

wphyojpl commented Jun 10, 2025

I guess that works too. I'm not sure it will cause problems when the collection is created again.

@ngachung ngachung self-requested a review June 25, 2025 21:06
@ngachung ngachung merged commit 7ae5ffb into develop Jun 25, 2025
1 check passed
@wphyojpl wphyojpl deleted the collection-delete branch June 25, 2025 21:07
@ngachung ngachung mentioned this pull request Jun 25, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ngachung ngachung ngachung approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@wphyojpl @ngachung